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. . .SPECIFICATION 


animal or 


mechanistic, form in 



a computer-generated 
virtual environment. 

The form taken by the virtual environment will depend on the 
intended use of the system and may, for. example, comprise the... 

..urban or surreal landscapes for games or other applications, around 
which environment a virtual body controlled by the user is moved . 
The form of the virtual body may vary as required by the application, 
from little ... 

..terms of available processor power for rendering of such models. In the 
following, the term " virtual environment " will be used f or such 
virtual landscapes or situations: the term "physical environment" will be 
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.example a room in which the user contacts the various hardware devices, 
such as a head - mounted display ( HMD ) , which provide the interface 
to the virtual environment . The terms "virtual body" and "physical 
body" should be construed accordingly. 

One example of a . . . such an apparatus is illustrated in Figure 1 which 
shows a plan" view comprising a stereoscopic HMD unit 10 driven by a 
CPU 12 to display images of a virtual environment from a movable 
viewpoint. The images of the environment are based on data from an... 

.the virtual body within the environment (such as a hand or claw which 
may be moved by the user, and used to manipulate objects within the 
virtual environment ) are based on data in a read only memory 16 also 
coupled with "the CPU. . . 

.image, such as shown in Figure 2, of a hand or claw floating within the 
virtual environment . Whilst the button 20 on the control device 18 
is not depressed, the three-dimensional movements that the user makes 
with the control device 18 are replicated by the movements of the 
virtual hand 26 within the virtual environment . The movements of the 
HMD 10 are also replicated in changes to the location of viewpoint from 
which the image of the virtual environment is rendered, although it 
will be appreciated that this movement will be limited as physical... 
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English Abstract 

A Human-Computer Interface that comprises a set of sensprs, actuators and 
computer generated controls is characterised in that the controls are 
projected in three spatial dimensions by a three dimensional display 
device and are manipulated by means of an operator interacting with them 
by movements of the operator's body, these movements being detected using 
the sensor system, where characteristics of a displayed target volume 
change according to the movement made, and actuators worn on the body, 
react according to how the body interacts with the computer generated 
controls . 



French Abstract 

L 1 invention concerne une interface homme-machine comprenant une serie de 
capteurs, d ' actionneurs et de commandes controlees par ordinateur. Cette 
interface est caracterisee par le fait que les commandes sont projetee en 
trois dimensions spatiales par un dispositif d'affichage en trois 
dimensions et qu'elles sont manipulees par un operateur interagissant 
avec elles moyennant des mouvements de son corps. Ces mouvements sont 
detectes par le groupe-capteur, les caracteristiques d'un volume cible 
projete changent en fonction du mouvement effectue, et les actionneurs 
portes sur le corps de 1 ' utilisateur reagissent en 'fonction de 
1 1 interaction entre le corps et les commandes generees par 1 1 ordinateur . 

Legal Status (Type, Date, Text) 

Publication 20011129 Al With international search report. 

Publication 20011129 Al Before the expiration of the time limit for 

amending the claims and to be republished in the 
event of the receipt of amendments. 

Examination 20020214 Request for preliminary examination prior to end of 

19th month from priority date 



Fulltext Availability: 
Detailed Description 



Detailed Description 

which to work. To counter this, there are 3D lo visualisation systems, 
such as the CAVE system supplied by the Virtual 

Environment Lab, Virginia Tech, Blacksburg VA 24081, which provide the 
operator with a true 3D image... 

...the walls of which are 

projected images that are viewed as 3D using a suitable headset . These 
systems, although useful for visualisation, are not currently suited for 
designo According to the... 

...controls are projected in three spatial 

dimensions by a three dimensional display device and are manipulated by 
means of an " operator interacting with them by movements of the 
operator 1 s 

body, these movements being detected using the sensor system, where 
characteristics of a displayed target volume change according. . . 
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Detailed Description 

Claims 

Fulltext Word Count: 6396 
English Abstract 

The present invention provides a modular, flexible system for deploying 
multiple video perception technologies. The telepresence system of the 
present invention is capable of allowing an operator to control multiple 
mono and stereo video inputs in a hands-free manner. The raw data 
generated by the input devices (22) is processed into a common zone 
structure that corresponds to the commands of the user, and the commands 
represented by the zone structure are transmitted to the appropriate 
device. The modularized approach permits input . devices to be easily 
interfaced with various telepresence devices. Additionally, new input 
devices and telepresence devices are easily added to the system and are 
frequently interchangeable. The present invention also provides a modular 
configuration (36) component that allows an operator to define a 
plurality of views each of which defines the telepresence devices to be 
controlled by a particular input device. The present invention provides a 
modular flexible system for providing telepresence for a wide range of 
applications. The modularization of the software components combined with 
the generalized zone concept allows the systems and methods of the 
present invention to be easily expanded to encompass new devices and new 
uses. 

French Abstract 

L 1 invention concerne un systeme modulaire souple permettant de deployer 
une multitude de technologies de perception video. Le systeme de 
telepresence est en mesure d f assurer a un utilisateur la commande par la 
technique main libre de multiples entrees video mono et stereo. Les 
donnees brutes generees par les disposiitifs de saisie (22) sont traitees 
et converties en une structure de zone commune correspondant aux 
commandes de 1 ' utilisateur et ces commandes representees par la structure 
de zone sont transmises au dispositif adequat . La modularite permet 
1 ' interf acage aise des dispositifs de saisie avec d'autres dispositifs de 
telepresence. Par railleurs, ces nouveaux dispositifs de saisie et de 
telepresence sont facilement ajoutes au systeme et souvent remplaces. 
L* invention concerne egalement un element de configuration modulaire (36) 
permettant a 1 ' utilisateur de definir une pluralite de vues, chacune 
definissant les dispositifs de telepresnece commandes par un dispositif 
de saisie particulier. L 1 invention concerne en outre un systeme modulaire 
souple servant a assurer la telepresnece pour une grande variete 
d' applications . La modularisation des elements logiciels combinee a la 
notion de zone generalisee facilite l 1 evolution des systemes et procedes 
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precites qui peuvent ainsi prendre en charge de nouveaux dispositifs et 
*de nouvelles utilisations. 

Legal Status (Type, Date, Text) 

Publication 20001012 Al With international search report. 

Publication 20001012 Al Before the expiration of the time limit for 

amending the claims and to be republished in the 
event of the receipt of amendments. 

Fulltext Availability: 
Detailed Description 

Detailed Description 

. . . s head. In the case of a zoom camera, for example, the forward and 
backward movement of an operator 's head may be 5 interpreted as a 
command to cause a camera to zoom in or out. Alternatively, the forward 
and backward movement of an operator 's head could also be interpreted 
as a command to physically move the camera either forward or 
backward. The actual implementation can be configured as needed. 

However ... 

...1 00 is interpreted as no motion and is present essentially to ensure 
that inadvertent movements are not interpreted as a movement command 
. Thus, when an operator is using a headset , the operator f s head 
does not need to be held perfectly still and slight head movements will 
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Claims 

Fulltext Word Count: 9803 
English Abstract 

This invention relates to a unique control approach that provides a 
means for manual or automated switching of narrowband controllers in 
personal active noise reduction (ANR) systems. A second related 
innovation links the human auditory system's physiological features to 
the design of a specific switching controller that implements ANR 
technology in audiometric testing. The design incorporates a speaker (7) 
as an actuator that modifies the microphone (9) sensor to reject the 
ambient noise. A summing junction (11) is included prior to speaker 
amplifier to add the controller (10) output to the prefiltered (5) 
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audiometry (4) test signal. For a given control objective of narrowband 
acoustic disturbance rejection, an analog or digital feedback controller 
can be designed to accomplish this goal over the precise bandwidth of the 
disturbance (s) . The new switching controller which is the key element of 
this invention requires that multiple feedback controllers be integrated 
with the personal ANR system, each one designed for maximum 
suppression/minimum spillover over specific and different narrowbands of 
frequencies . 

French Abstract 

L' invention porte sur un concept unique de commande relatif a un moyen 
de commutation manuelle ou automatique d'un controleur de bandes etroites 
de systemes personnels de reduction active du bruit (ANR) . Une deuxieme 
innovation associee relie les caracteristiques physiologiques du systeme 
auditif de 1 ' homme a un controleur specif ique de commutation mettant en 
oeuvre la technique ANR pour des tests audiometriques . Ledit controleur 
comporte un haut-parleur (7) servant a modifier le detecteur du 
microphone (9) pour rejeter le bruit ambiant . Une liaison (11), disposee. 
avant 1 1 amplif icateur de sommation du haut-parleur ajoute le signal de 
sortie du controleur (10) au signal prefiltre (5) du test d 1 audiometrie 

(4). Pour un objectif donne de commande de la rejection des perturbations 
acoustiques a bande etroite, on peut concevoir un controleur analogique 
ou numerique a retroaction atteignant ce but pour la bande precise des 
perturbations. Le nouveau controleur de commutation qui est 1' element 
clef de 1' invention requiert 1 1 integration au systeme ANR personnel de 
plusieurs controleurs a retroaction dont chacun est concu pour une 
suppression maximale/et un debordement minimal sur des bandes etroites de 
frequences specif iques et differentes. 

Fulltext Availability: 
Detailed Description 

Detailed Description 

... be automatically determined based on 

quantitative data or they may be determined by the end user 
as in the application described next. 

Active noise control headsets used for hearing 
protection and improved speech intelligibility in high noise 
environments are fairly common. . . 
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Detailed Description 
Claims 

Fulltext Word Count: 5654 
English Abstract 

A method and apparatus are provided for providing user-directed 
operation of" a virtual manipulator, such as a simulated hand or claw (30) 
within a computer-generated virtual environment. The environment 
comprises one or more objects (32) which may be held, released or 
impelled by the manipulator under user control. Movement of the virtual 
manipulator is directed by a suitable hand-held device operated by a 
user, with a simple control directing opening or closing of the claw 
(30) . In operation, actions such as throwing, catching or punching an 
object (32) are partially automated, with the decision as to the action 
initiated being determined by the separation (D) between object and 
manipulator, and the current and previous states of each as recorded in 
the form of respective state variables. A method for the derivation of an 
applied velocity for the object acted upon is also described. 

French Abstract 

Cette invention se rapporte a un procede et un appareil permettant a un 
utilisateur de gerer 1' action d'un manipulateur virtuel, tel qu'un gant 
ou une pince simulee (30) a 1 1 interieur d'un environnement virtuel 
produit par ordinateur. Cet environnement contient un ou plusieurs objets 
(32) qui peuvent etre saisis, relaches ou propulses par le manipulateur 
sous le controle de 1 1 utilisateur . Les mouvements du manipulateur virtuel 
sont geres par un dispositif manuel approprjle actionne par 1 1 utilisateur, 
lequel gere par une simple commande l'ouverture ou la fermeture de la 
pince (30) . En mode d 1 utilisation, les actions telles que lancer, 
attraper ou frapper un objet (32) sont partiellement automatisees, la 
decision concernant l 1 action a entreprendre etant determinee par la 
separation (D) entre 1' objet et le manipulateur, et les etats presents et 
passes de chacune de ces actions etant enregistres sous la forme de 
variables d* etats respectifs. Cette invention presente egalement un 
procede permettant de deriver la vitesse appliquee a 1' objet sur lequel 
porte 1' action. 

Fulltext Availability: 
Detailed Description 

Detailed Description 

... users limb and reproduce such movements within a virtual environment. 

The form taken by the virtual environment will depend on the intended 
use of the system and may, for example, comprise the... 
...or surreal landscapes for games 1 5 or other applications, around which 
environment a virtual manipulator controlled by the user is moved 
. The modelled form of the virtual manipulator may vary as required by 
the application (along with the form of the virtual objects... 

. . . pads of an 

octopus) to mechanical equivalents and specialised tools. Whatever the 
particular form, the manipulator must be able (under direct or indirect 
user 

control ) to hold an object (such that it may be moved or stopped from 
movement by the manipulator), it must be able to release an object... 

...to an object (pushing) without having first to hold it. In the 

following, the term " virtual environment " will be used for all such 
computer-generated virtual landscapes or situations: the term "physical 



...example a room in which the user contacts the various hardware devices, 
such as a head - mounted display ( HMD ) , which provide the interface 
to the 
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virtual environment . The terms "virtual body" and "physical body" 
should be construed accordingly. 

One example of a... claw) floating within the virtual 1 5 environment. 
Other objects may be present in the virtual environment , such as 
balls 32 and 32 ' , which the user may be intended to catch or throw. The 
three 

dimensional * movements that the user makes with the control device 
18 are 

replicated by the movements of the virtual manipulator 30 within the 
virtual 

environment . The movements of the HMD 10 are also replicated in 
changes 

to the location of the viewpoint from which the image of the virtual 
environment 

is rendered. The movements are implemented in successive iterations of a 

simulation loop by the CPU 12. In each iteration, positional changes of 
the 

manipulator 30 (and. . . 
. .determined from the 

control 18, positional changes of any of the objects 32 within the 
virtual environment (whether directly or indirectly the result of user 
action, or as a result of the complete or partial autonomy of those 
objects within the confines of the simulation program) are calculated, 
and a revised image of the virtual 

environment (also taking account of any changes to the user viewpoint 
as 

indicated by signals from HMD 10) is rendered and presented to the 
user. As this movement will be limited due... 

. . .movement being constrained by the apparatus, a means to enable more 
extensive movement within the virtual 

environment may be provided. Such a means is described in our pending 
unpublished United Kingdom patent... 
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Claims 

Fulltext Word Count: 2575 
English Abstract 

An apparatus is provided for modelling and interacting with a virtual 
environment. An image of the environment, and a representation of the 
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When the button (20) is actuated, uie u» . „hiftina the location 

(18) are instead converted to navigational commands shifting tne loca 
of the user's virtual presence around the virtual environment. 

^Sn^S^concerne un dispositif servant a ^^^rr"' 
Srefe^Lfof de^^prefencrde 1 • uti?isateur dans ledit environnement, 

Sl^ir.^.^t^'S fcrLlLr^co^gue 4 ,!"! . 

e r e vu d'une unite (18) qu'il tient a la mam et qui 

^^1=^^X1^ Jonf 
Sie^ 

de Ration, ce qui deplace 1 • emplacement de l. presence virtuelle de 
l'utilisateur autour de 1 ' environnement virtuel. 

Fulltext Availability: 
Detailed Description 

"^^^echSisri^'xorm in a computer-generated virtual environment. 

1 0 The form taken by the virtual environment will depend on the 
intended use of the system and may, for example, comprise the... 

.urban or surreal landscapes for games or other applications, around 
which environment a virtual body controlled by the _ 
user is moved . The form of the virtual body may vary as required by, 

the 

1 5 application. . . 

...example a room in which the user contacts the various 

hardware devices, such as a head - mounted display ( HMD ), »^h 
provide the interface to the virtual environment . The terms virtual 
body" and "physical body" should be construed accordingly. 

One example of a... such an apparatus is illustrated in Figure 1 which 
:S S com^sing a stereoscopic HMD unit 10 driven by a CPU 12 to 
Sage^of a virtual environment from a movable viewpoint. The images 
of the 

environment are based on data from an... 

the virtual body within the environment (such as a hand or claw which 
"mav be moved by the user and used to manipulate objects within ^ 
the visual environment ) are based on data in a read only memory 16 

also 

coupled with the CPU. . . 
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Fulltext Availability: 
Detailed Description 
Claims 

Fulltext Word Count: 22237 
English Abstract 

A visual display device is provided for delivering a generated image, 
preferably combinable with environment light, to the eye of a user. In 
one embodiment an image generator is masked by at least two masks (222, 
224) to provide for a high quality image without waste. In one 
embodiment, a removably mounted shield (302) or activatable device 
converts the apparatus from a see-through device to an immersion device 
and back again. A tracker (100) for outputting an indication of the 
orientation,- attitude and/or position of a head-mounted display (HMD) may 
be provided. The tracker (100) can be configured so that it is 
incorporated in the HMD housing and/or can be easily decoupled from the 
HMD, so that the HMD can be used without the tracker (e.g. for 'watching 
movies) . Preferably, the tracker provides pass-through of signal to the 
HMD and, when coupled to the HMD, only a single cable (134) connects the 
HMD-tracker combination to the host computer (510) . In one embodiment, 
the tracker uses magnetic sensors (542a, 542b, 542c) . In another 
embodiment, one or more inertial sensors, such as a rate gyro (52) and/or 
accelerometers are used (562a, 562b) . 

French Abstract 

Dispositif de visualisation destine a mettre une image projetee, de 
preference une image combinable avec la lumiere du jour, a la portee des 
yeux d'un utilisateur. Dans un mode de realisation, une visiere (112) 
protege des reflets et maintient les elements optiques dans l f alignement 
voulu. Dans un mode de realisation, un diffuseur d'images est pourvu d'au 
moins deux masques (222, 224) pour pouvoir fournir des images de bonne 
qualite, sans pertes . Dans un mode de realisation, un pare-lumiere 
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nn ,| nn , m disoositif pouvant etre active permet de faire 

des capteurs magnetiques (542a, 542b, 542c). Dans one autre mode de 
realisation, on fait appel a un ou plusxeurs cap ears t xels, tels 

qu'un gyrometre (52) et/ou des accelerometres (562a, 562b). 



Fulltext Availability: 
Detailed Description 

Detailed Description 

has increased ease of 

S'X "e3 i of t a°"h.;d Pai »ou» d ti he « . it is de S i te d to ccUec, 

be 

location and/or movement or me user a uca „___. fnr 

used to control the image generated to the eyes of the user or a 
number of purposes, such, .this information can be used to control the 
characteristics of the image shown in a head - mounted display or 
Sher SISS reality device, such as to produce changes in the image 
Sat would correspond to movement of the user's head. For example, in a 
fUght sLu^of program, when the user's head rotates from a straight 
ahead position to a 90... 
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location on the display with respect to a viewpoint for the display, 

and a selection stage for determining which of the plurality of display 
JESS P°osftion o^haf objS Jo^SS S ^ 



^SySteS^rii-u.lisation graphique assurant le -ndu ^es *ur^es 
surfaces entourant 1 ' utilisateur, par un precede de re jju J Pnoritea . 
Qv^t-Pme a acces aux images visualisees destinees a la fenetre 
S vt en Silisant un pipeline de recalcul d'adreaae .On a acces 
aux donnees d' image en determinant une orientation de rotation de 1 image 

U f e en adressant les donnees d' image stockees en fonction de 

I-oSeSSion de rotation determinee. Afin d'afficher les donnees 
relatives aux pixels, on calcule un premier vecteur indiquant 
1- emplacement d'un pixel sur l'affichage par rapport a un point 
o^ser^tion de l'affichage, on calcule un deuxieme vecteur xndxquant 
une orientation spatiale de l'affichage, on combine lesdits premier et 
Seuxie^ vecteurs P en un vecteur resultant, on ^e^ne un^oxnt 
d' intersection du vecteur resultant avec une surface notionnelle 
essentielleient fermee lors de sa projection a partir d'un point de 
nroiertion interne fixe, et on mappe ledit point d' intersection sur les 
donneea S'imSge stockees, de maniere a adresser les donnees d' image pour 
Vaffichage audit emplacement de pixel. Ce systeme comprend au moins un 
module de rendu graphique qui genere des donnees d' image correspondent 
Tut nforma"ons 9 d'affichage d'objet stockees relatives |Jjet. d evant 

etre visualises par le systeme, un ensemble de memoires d affichage qui 
stockent les donnees d' image genereespar le(s) module (s de rendu 
grapnique 7 et un etage de Selection qui determine laquelle des memoires 
ri'affichaae doit stacker les donnees d' image relatives a un ob^et 
particuUer en fonction de la position relative de cet objet par rapport 
a l'utilisateur. Les memoires d'affichage sont mises a ]our 
periodiquement avec les donnees d' image provenant du module de rendu, et 
la periode entre chaque mise a jour est differente d'une memoire 
d'affichage a 1' autre. 

Fulltext Availability: 
Detailed Description 

Detailed Description 

" Imaqe e ov^iaying or image composition is a technique often used by 
• Scost non' head - mounted display 9«phics systems such as video 

very well with conventional display controllers m virtual reality 
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systems as. user head rotations invalidate all display memories 
including those containing background images. This mass invalidation 
occurs because all... become invalid when the viewport mapping changes 
The only exception to this is when providing stereoscopy for large 
orientation changes. Some re-rendering of close objects may be 
required for an accurate stereoscopic view of the scene 

Figure 4 shows a block diagram of a known image generation. . . 



August 20, 2003 



21/5, K/l (Item 1 from file: 348) 

DIALOG (R) File 34 8: EUROPEAN PATENTS 
(c) 2003 European Patent Office. All rts. reserv. 



00537294 

Image processing unit and method for executing image processing 
Verfahren und Einrichtung zur Bildverarbeitung 
Methode et dispositif de traitement d' image 

PATENT ASSIGNEE: 

FUJITSU LIMITED, (211460), 1015, Kamikodanaka, Nakahara-ku, Kawasaki-shi, 
Kanagawa 211, ( JP) , (applicant designated states: DE; FR; GB) 
INVENTOR: 

Murakami, Koichi , c/o Fujitsu Limited, 

Kawasaki-Shi, Kanagawa 211, (JP) 
Hirota, Katsuhiko, c/o Fujitsu Limited, 

Kawasaki-Shi, Kanagawa 211, (JP) 
Nishiyama, Soichi, c/o Fujitsu Limited, 

Kawasaki-Shi, Kanagawa 211, (JP) 
LEGAL REPRESENTATIVE: 

Skone James*, Robert Edmund (50281), GILL JENNINGS 

7 Eldon Street, London EC2M 7LH, (GB) 



1015 Kamikodanaka, 
1015 Kamikodanaka, 
1015 Kamikodanaka, 



Nakahara-ku, 
Nakahara-ku, 
Nakahara-ku, 



PATENT (CC, No, Kind, Date) 



EP 502643 A2 920909 
EP 502643 A3 930721 
EP 502643 Bl 960605 
EP 92301570 920225; 



& EVERY Broadgate House 
(Basic) 



APPLICATION (CC, No, Date) : 
PRIORITY (CC, No, Date) : JP 9139732 910306 
DESIGNATED STATES: DE; FR; GB 
INTERNATIONAL PATENT CLASS: G09G-005/00 
CITED PATENTS (EP A): GB 2201069 A; US 4348186 A 
CITED REFERENCES (EP A) : 

SCIENTIFIC AMERICAN vol. 257, no. 4, October 1987, NEW YORK US pages 82 
90 J.D. FOLEY interfaces for Advanced Computing 1 ; 



ABSTRACT EP 502643 A2 

An image processing unit generates a virtual environment or an 
artificial reality by displaying a necessary image in accordance with 
movement of a view direction of a person on a head mounted display and 
■comprises a sensor (10), an image processing portion (11), and. a display 

(13) . The sensor (10) detects a view direction and generates a view 
vector, and the image processing portion (11) comprises a view direction 
movement discriminator (14), a priority degree allocator (15), and a 
partial update portion (16). The view direction movement discriminator 

(14) calculates a view direction movement vector from the difference 
between the latest two fetched view vectors, the priority degree 
allocator (15) generates a projection vector by projecting the view 
direction movement, determines a priority degree at each divided section 
of the display screen to be higher when the projection vector approaches 
and to withdraw from the centre of the display screen to a periphery 
along the projection vector, and the partial update portion (16) updates 
the display of the display screen in each divided section in accordance 
with the priority degree, (see image in original document) 
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INTERNATIONAL PATENT CLASS: G09G-005/00 

"...SPECIFICATION of a view direction of a person mountining the same. 

In the system generating the virtual environment or the artificial 
reality, regarding the sense of sight, an image processing unit generates 
the necessary stereo image using a computer graphic technique, the image 
is displayed on a head mounted display mounted on the head of a 
user , and the image is updated in every instant in response to the 
movement of the... of this invention is explained with reference to the 
drawings . 

In the system generating the virtual environment or the artificial 
reality, regarding a sense of sight, as shown in Fig. 2, an... 

...the necessary stereo image using a computer graphic technique, the image 

is displayed, on a head mounted display 3 mounted on the head 2 of a 

user , and the image is updated in every instant in response to the * 
movement of the... 

...SPECIFICATION of a view direction of a person mounting the same. 

In a system generating the virtual environment or the artificial 
reality, regarding the sense of sight, (see for example documents GB-A 

...the necessary stereo image using a computer graphic technique, the 
image is displayed on a head mounted* display mounted on the head 
of a user , and the image is updated every instant in response to the 
movement of the properly ... of this invention is explained with reference 
to the drawings. 

In the system generating the virtual environment or the artificial 
reality, regarding a sense of sight, as shown in Fig. 2, an... 

...the necessary stereo image using a computer graphic technique, the 
image is displayed on a head mounted display 3 mounted on the head 
2 of a user , and the image is updated in every instant in response to 
the movement of the. . . 
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Detailed Description 

Claims 

Fulltext Word Count: 15372 
English Abstract 

An overall hardware configuration that produces an enhanced spatial 
television-like viewing experience is disclosed (Fig, 1) . Unlike normal 
television, with this system the viewer is able to control (remote 
control) both the viewing direction and relative position of the viewer 
with respect to the movie action (Fig. 6) . In addition to a specific 
hardware configuration (SO, SI, S2, S3), this patent also relates to a 
new video format which makes possible this virtual reality like 
experience (Wireless HMD) . 

French Abstract 

L' invention concerne une configuration materielle globale, qui cree une 
forme amelioree de visualisation spatiale, similaire a la television. 
Contrairement a la television classique, ce systeme permet au 
telespectateur de commander ( telecommander) la direction de visualisation 
et la position relative du telespectateur par rapport a 1' action du film 
(fig. 6). Outre cette configuration materielle specifique (SO, SI, S2, 
S3), la presente invention porte egalement sur un nouveau format video 
qui permet de realiser cette experience virtuelle proche de la realite 
(HMD sans fil) . 

Legal Status (Type, Date, Text) 

Publication 20030103 Al With international search report. 
Publication 20030103 Al With amended claims. 

Examination 20030515 Request for preliminary examination prior to end of 

19th month from priority date 

...International Patent Class: G09G-005/00 
Fulltext Availability: 
Claims 

Claim 

such that the horizontal direction of view within the 360 degree by X 
degree vertically " virtual environment " is dynamically controllable 
by the user at runtime (while the images being displayed) . 28) An 
electronic device as described in claims... 

...azimuth and elevation of the viewport within the 360 degree horizontal 
by X degree vertical " virtual environment " is dynamically 
controllable by the user at runtime (while the images being displayed) . 
29) An electronic device as described in claims ... described in claims I 
I- 16, 26-29, in which the virtual camera position within virtual 
environment " (i.e. the viewpoint of the viewer) is dynamically 
controllable by the user at runtime (while the images are being 
displayed). 31) An electronic device as described in... which is 
controlled by the direction of the viewport within the 360 degree Image 
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Sphere (" virtual environment " ) . 34) An electronic device as described 
in claims I I - 16, 26-3 1, in ... direction of the viewport within the 360 
degree Image Sphere and virtual position within the " virtual 
environment ". 35) An electronic device as described in claims 11-16, 
26-34, which... 41, which converts the three-dimensional objects into a 
universal graphics description language" such as VRML or other 
appropriate language for storage or live 
C" t, C@ 

transmission and subsequent decoding ... composite image" before displaying 
on the output device (such as a flat screen display or HMD ). 47) An 
electronic device as described in claims 43-45, which displays imagery as 
described. .. composite image" before displaying on the output device (such 
as a flat screen display or HMD ). 48) An electronic device as described 
in claims 47, which in addition to using the .. .movement in the azimuth of 
the camera (of the viewer in the case of an HMD based camera assembly) 
as shown in figs. 16,19 and more completely described in paragraphs 
viewer in the case of an HMD based camera assembly) as shown in figs. 
26,28 and more completely described in paragraphs... 

.movement in the roll of the camera (of the viewer in the case of an HMD 
based camera assembly) as 
shown in figs. 26,29 and more completely described in paragraphs ... the 
physical (spatial) position of the camera (of the viewer in the case of 
an HMD based camera assembly) in either or any combination of the X, Y 
or Z axes ... controllable on-axis light sources which are synchronized to 
the video capture rate of the HMD based or remote panoramic cameras to 
improve the ability of the system to correctly identify ... controllable 
on-axis light sources which are synchronized to the video capture rate of 
the HMD based or remote panoramic cameras to improve the ability of the 
system to correctly identify HMD based or remote panoramic cameras to 
improve the ability of the system to correctly identify. . . 
.optical beacons" (controllable light sources which are synchronized to 
the video capture rate of the HMD based or remote panoramic cameras) 
such that pulse timing, color of light and/or combinations of these are 
used to transmit the "real world" coordinates of the beacon to the HMD 
or remote panoramic camera to determine absolute ancyular/spatial 
references for ...a number of "bi-directional infrared beacons" which 
communicate a unique ID code between the HMD and the beacon such that 
this calibration) An electronic device which utilizes a single optical... 
as described in claims 11-17, in which the -viewpoint within the 3D " 
virtual environment " (i.e. the viewpoint of the viewer within a 3D 
virtual space that contains the objects that comprise the image) is 
dynamically controflable by the user at runtime (while the images are 
being displayed) . 32) An electronic device as described in claims 11-17, 
in which the viewpoint within the 3D " virtual environment " (i.e. the 
viewpoint of the vieNver within a 3D virtual space that contains the... of 
the viewport within the panorama and virtual position of the viewpoint 
within the 3D " virtual environment ". 36) An electronic device as 
described in claims 11-17 which is capable of combining ... which converts 
the three-dimensional objects into a 44 universal graphics description 
language" such as VRML or other appropriate language for storage or 
five transmission and subsequent decoding into graphical imagery... 
composite image" before displaying on the output device (such as a flat 
screen display or HMD ) . 47) An electronic device which incorporates a 
combination of Frequency Compression and Expanding (Companding) in... 
movement in the azimuth of the camera (of the viewer in the case of an 
HMD based camera assembly) as shown in figs. 16, 17, 23 and more 
completely described in the camera (of the viewer in the case of an HMD 
based camera assembly) as shown in figs. 16, 180 23 and more completely 
described in . . . 

.movement in the rofl of the camera (of the viewer in the case of an HMD 
based camera assembly) as shown in figs. 16, 19, 23 and more completely 
described in... the physical (spatial) position of the camera (of the 
viewer in the case of an HMD based camera assembly) in either or any 
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combination of the X, Y or Z axes ... controllable on-axis fight sources 
which are synchronized to the video capture rate of the HMD based or 
remote panoramic cameras to improve the ability of the system to 
correctly identify ... controllable on-axis light sources which are 
synchronized to the video capture rate of the HMD based or remote 
panoramic cameras to improve the ability of the system to correctly 
identify rate of the HMD based or remote panoramic cameras to improve 
the ability of the system to correctly identify... 

...optical beacons" (controllable fight sources which are synchronized to 
the video capture rate of the HMD based or remote panoramic cameras) 
such that pulse timing, color of light and/or combinations... 
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English Abstract 

A virtual simulation system generates an image of a virtual control on a 
display that may be a heads-up-display in a vehicle. The system uses 
three-dimensional range finding data to determine when a user is 
sufficiently close to the virtual control to "manipulate" the virtual 
control. The user "manipulation" is sensed non-haptically by the system, 
which causes the displayed control image to move in response to user 
manipulation. System output is coupled, linearly or otherwise, to an 
actual device having a parameter that is adjusted substantially in 
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real-time by user-manipulation of the virtual image. System generated 
displays can be dynamic and change appearance when a user's hand is in 
close proximity displays can disappear until needed, or can include menus 
and icons to be selected by the user who points towards or touches the 
virtual images. System generate images can include representation of the 
user for use in a training or gaming system. 

French Abstract 

L' invention porte sur un systeme virtuel de simulation elaborant 1 ' image 
d'une commande virtuelle sur un ecran pouvant etre le dispositif tete 
haute d f un vehicule. Ledit systeme utilise des donnees de telemetrie en 
3D pour determiner quand 1 1 utilisateur est suffisamment rapproche de la 
commande virtuelle pour pouvoir la "manipuler " . Les "manipulation" de 
1 1 utilisateur sont detectees non haptiquement par le systeme, ce qui 
provoque le deplacement des images de commande en reponse a ladite 
manipulation. Les signaux de sortie du systeme sont couples, lineairement 
ou autrement, a un dispositif reel presentant un parametre s'ajustant 
sensiblement en temps reel lors de la manipulation de 1 1 image virtuelle 
par 1 1 utilisateur . Les images produites par le systeme peuvent etre 
dynamiques et changer d'apparence lorsque l ! une des mains de 
1 1 utilisateur se trouve a proximite immediate. Les images peuvent 
disparaitre jusqu'a ce qu'un les utilise, ou peuvent comprendre des menus 
et des icones selectionnables par 1 ' utilisateur en pointant sur une image 
virtuelle ou en la touchant . Les images produites par le systeme peuvent 
comprendre une representation de 1 1 utilisateur utilisable dans des 
systeme de jeu ou d 1 entrainement . 
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add to the cost of a computer simulated system, and can be cumbersome 
to the user . Not only is freedom of 1 5 motion restricted by such 
sensor-implemented devices, but... 

...sized gloves for adults, 

medium-sized" gloves, small-sized gloves, etc. Further, only the one user 

wearing the body suit, glove, helmet, goggles can utilize the virtual 
system; onlookers for example see essentially nothing. An onlooker not 
wearing such sensor-laden garments cannot participate in the virtual 
world being presented and cannot manipulate virtual objects. 

U.S. patent no. 5,168,531 to... 
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English Abstract 

A vehicle simulator (40) having a projection system (42) for projecting a 
generated scene (44) and symbology (46) onto a common viewing screen 
(48) . Tracking apparatus (50) is provided for providing position and/or 
angular orientation signals representative of the eye (19) of a person 
viewing the projected scene and symbology. An image generation system is 
included for generating the scene and symbology for the projection 
system, such scene and symbology being generated as a function of the 
position and/or angular orientation signals provided by the tracking 
apparatus. Such symbology is representative of symbology produced by a 
head-up display. 

French Abstract 

La presente invention concerne un simulateur de vehicule equipe d'un 
systeme de projection concu pour projeter une scene et la symbologie 
generee sur un ecran de visionnage commun. Un appareil de poursuite 
permet de fournir des signaux de position et/ou d ' orientation angulaire 
caracteristiques de l'oeil d'une personne visionnant la scene projetee et 
sa symbologie. Un systeme de generation d' image permet de generer la 
scene et la symbologie destinees au systeme de projection, ces scenes et 
symbologie etant generees en fonction des signaux de position et/ou 
d 1 orientation angulaire fournis par 1' appareil de poursuite. Une telle 
symbologie est representative de la symbologie produite par un systeme de 
vision tete haute. Le systeme de generation d 1 image genere la scene et la 
symbologie avec des caracteristiques qui simulent les effets de 
vignettage et d 1 occultation caracteristiques de la vision tete haute. La 
scene par la fenetre et la symbologie sont projetees sur un ecran et 
co-localisees alors que le generateur damage, la base ou la position de 
l'oeil de 1 ' utilisateur produisent la symbologie de facon a simuler les 
effets de collimation, de vignettage et d ' occultation du type optique de 
vision tete haute .utilise dans le vehicule qui est simule. 
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... arrow 4 6) onto a common viewing screen 48. Other methods of presenting 
images to the user may also be used, including a CRT display, an LCD 
display, various shaped screens with varying surfaces, viewed directly or 
through optical systems, or helmet -mounted displays. Tracking apparatus 
50 is provided to give position and/or angular orientation signals... 

. . .position of the eye 19 of a person, here a pilot being trained in the 
simulator 1 0, and viewing the projected scene 44 and the symbology 46 
on screen 48 . . . 
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English Abstract 

A new tracking technique is essentially "sourceless" (30) in that it can 
be used anywhere with no set-up, yet it enables a much wider range of 
virtual environment-style navigation and interaction techniques than does 
a simple head-orientation track. A sourceless head orientation tracker 
(30) is combined with a head-worn tracking device (12) that tracks a 
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hand-mounted 3D beacon (14) relative to the head (16). The system 
encourages use of intuitive interaction techniques which exploit 
proprioception . 

French Abstract 

La nouvelle technique de poursuite de l 1 invention est sensiblement <= 
sans source >= (30) en ce qu'elle peut etre utilisee sans initialisation 
et qu'elle permet quand-meme 1 1 utilisation d'une plage sensiblement plus 
grande de techniques de navigation dans un environnement virtuel et 
d 1 interaction qu'avec une simple poursuite de 1 ' orientation de la tete. 
Un systeme de poursuite de 1 1 orientation de la tete (30) sans source est 
combine a un dispositif de poursuite (120) porte sur la tete, qui 
poursuit une balise 3D (14) montee sur' la main, par rapport a la tete 
(16). Ledit systeme favorise 1 ' utilisation de techniques d 1 interaction 
intuitives exploitant la proprioception. 
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representative of the motion of the hand relative to tile head. 

Figure 5 illustrates a user wearing a portable VR tennis game or 
training system. The computer and batteries are contained in backpack 
502. which is cabled to HMD 500 to which - 18are mounted inertial 
sensors 506 and ultrasonic transducers 5 10. He... 
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English Abstract 

A 3D navigation system employs a complementary pair of emitter/detector 
units, one (10) worn on a part of a user's body (e.g., the head) and the 
other (20) mounted in a stationary position on a display monitor facing 
the user. Each unit has an array of photovoltaic (PV) cells (10b, 20b) 
centered on a normal axis and an infrared diode (10a, 20a) for emitting a 
light beam on the normal axis. Each photodetectors array receives the 
infrared beam from the other unit and provides output signals 
representing the light intensities of the received beam on the PV cells. 
The PV cell response signals of the complementary units are used together 
to calculate the position and angular orientation of the user-worn unit 
relative to the stationary unit. The position and angle values are used 
by a software device driver installed on a computer to control 3D 
navigation functions of the associated application program on the 
computer. The system can be used for playing interactive 3D games, and 
also used for hands-free control in a wide range of other applications 
and environments, such as to control flight motions in a simulator or 
cockpit of an aircraft, or the command center of a naval vessel, 
spaceship, land vehicle, etc. for disabled persons to provide hands-free 
input to a computer, monorized wheelchair, prosthetic device etc, and for 
controlling various functions in an industrial environment, such as 
guiding machine-assisted movement, or in CAD design, architectural, 
medical graphics, virtual reality, and other applications. 

French Abstract 

Un systeme de navigation tridimensionnelle utilise deux blocs 
complementaires detecteur/emetteur , l ! un (10) se situant sur une partie 
du corps d'un utilisateur (par ex., la tete) et 1'autre (20) etant fixe 
dans une position fixe sur un moniteur d'affichage face a 1 1 utilisateur . 
Chaque bloc comprend un reseau de cellules photovoltaiques (PV) (10b, 
20b) centrees sur un axe perpendiculaire et une diode infrarouge (10a,. 
20a) servant a emettre un faisceau de lumiere sur ledit axe. Chaque 
reseau de photodetecteurs recoit le faisceau a partir de l 1 autre bloc et 
produit des signaux de sortie representant les intensites de lumiere du 
faisceau recu sur les cellules PV. Les signaux de reponse des cellules PV 
des blocs complementaires sont utilises pour calculer la position et 
1 'orientation angulaire du bloc non porte par 1 1 utilisateur par rapport 
au bloc fixe. Les valeurs d' angle et de position sont utilisees par un 
pilote de dispositif de logiciel installe sur un ordinateur afin de 
commander des fonctions de navigation tridimensionnelles du programme 
d' application associe. On peut utiliser ledit systeme pour jouer a des 
jeux interactifs tridimensionnels, et dans le cadre de commande mains 
libres d'une large variete d'autres applications et d ' environnements, 
tels que la commande des mouvements de vols dans un simulateur de vol ou 
un cockpit d' avion, ou le centre de commande d'un navire, d'un vaisseau 
spatial, d'un vehicule terrestre, etc. Ce systeme peut egalement etre 
utilise par des personnes handicapees de maniere a leur fournir une 
entree mains libres a un ordinateur, avec une chaise roulante motorisee, 
un dispositif prothetique etc., et dans un environnement industriel pour 
commander diverses fonctions, telles que le guidage de mouvements 
assistes par machine, ou dans un modele CAAO, avec des graphiques 
medicaux et architecturaux, en realite virtuelle et dans d'autres 
applications . 
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^"^siSrgames^It may be used to control flight motions and display 
"views in a Simulator or cockpit of an aircraft or the command center 
of a naval vessel, spaceship, land vehicle, etc. Leaving the user s 
hands free allows for simultaneous control of other functions, such as 
manipulation of controls... 

for example, guiding the machine-assisted movement and orientation of a 
'"worUecewhlle leaving the operator -s hands free to manipulate process 
equipment applied to the workpiece (welding, painting, laminating, etc... 

...design, architectural, medical graphics, virtual reality, »d ^er 

commercial applications. For example, use of the headset with a virtual 
reality display can allow a 3D "tour" to be taken through construction 
sites, buildings, medical diagnostic images, and simulated or artificial 
environments . 

Other objects, features, and advantages of the present invention will be 
described in further detail... the average value. 

In FIG. 61 a screen-shot of the interface for a software simulation 
program is shown for modeling the solution inputs of the device A user 
can vary the angles and location of the headset unit. The cells are 
identified in the left-hand column as H1-H4 for the headset unit and 
M1-M4 for the monitor unit. The cell numbering proceeds from the top... 3D 
••tour 1 to be taken moving through 3D constructions, buildings, medical 
images, and simulated or artificial environments , with lifelike angle 
shifts of the user 's head simulating corresponding changes in view. 

The movable unit could also be mounted on... 
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9 A virtS computer monitor is described which enables instantaneous and 
intuitive visual access to large amounts of visual ^ata by providing the 
user with a large display (20) projected virtually m front of the user. 
The user wears a head-mounted display (26) or holds a portable display 
containing a head-tracker (28) or other motion tracker, w toget her 
allow the user to position an instantaneous viewport 22) P"^ ed the 
display at any position within the large virtual display (20) by turning 
to look in the desired direction. The instantaneous viewport (22) further 
includes a mouse pointer, which may be positioned by turning the user s 
• head or moving the portable display, and which may be further positioned 
using a mouse or analogous control device. A particular advantage of the 
virtual computer monitor is intuitive access to enlarged computer output 
for visually-impaired individuals. 



i-Jnv^ntion porte sur un ecran virtuel d'ordinateur permettant un acces 
visuel instantane et intuitif a une grande quantite de donnees visuelles 
presentees a 1 ' utilisateur sur un grand ecran (20) virtuel dispose face a 
lui L ' utilisateur utilise a cet effet un visiocasque (26) ou un 
afficheur portable lie a un casque-ecran (28) ou un suiveur de mouvement, 
qui conjointement permettent a 1 ' utilisateur de selectionner un secteur 
visuel instantane (22) correspondant a l'une des positions du grand ecran 
virtuel (20) en tournant la tete pour fixer la direction desiree. Le 
secteur visuel instantane (22) comporte en outre un curseur de souris 
pouvant se positionner par une rotation de la tete ou un deplacement de 
l'afficheur portable, ou a l'aide d'une souris ou d'un dispositif de 
commande. analogue. L'un des avantages particuliers de 1 ecran virtuel 
d'ordinateur reside dans la possibilite donnee a des malvoyants d acceder 
intuitivement a d'importante quantites de donnees fournies par un 
ordinateur . 

Main International Patent Class: G09G-005/00 
Fulltext Availability: 
Detailed Description 

Detailed Description . 

simulated objects in the environment using a control peripheral. 

In a virtual reality system, the user is "immersed" in a synthetic 
environment," in which virtual objects can be located anywhere m the 
user «s physical space. The user views these objects by wearing a head 
- mounted display ( HMD ) , which uses an optical system to cause a 
tiny display source such as a cathode... 

...crystal display to appear as a large display screen several feet in 
front of the user . Since the display source (or sources in the case or 
two eyes) is fixed to the user 's head, the display is viewable 
regardless of where the user points his line-of-sight . The user also 
wears a head - tracker , which senses the direction the user is 
facing, and sends this information to the host computer. The computer 
uses this data to generate graphics corresponding to the user s line or 
siqht in the virtual environment . This approach to human/computer 
interfaces was first conceived by Ivan Sutherland in 1966 for use in 
military simulators, and was first commercialized in the form of the 
Eyephone head - mounted display by 
VPL Research in the late 1980s, 
Prior art in this area includes a wide... 

patents describing low-vision aids, improved virtual reality systems and 
"components such as HMDs and head - trackers , but none which embody or 
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anticipate the present invention. 

, c i r-io-i^r, hmo imaaerv can be implemented analogously 

virtual •»""°™ e "*' /""Virtual computer desktop surrounding the 
^« £ :"nLn y cK°prov 1 de%i n u!taneour,cce S ? to a lar,er amount of visual 
data than is possible... 

currently provided on common computing platforrns such ^Macintosh and 
■"disSr/tHnlSf^r-ri^^^^itl^c^r^ni^vrrtual 
page, and thus enable... of the image. 

real objects can... 



user 
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^heaSfounted display system (10) and a method of ^^^,^^1 
within the system include two or more reconfigurable holographic opti cal 

HOEs may also be configured to perform sophisticated optical 

a Sogram (50, 52 and 54) that is sandwiched between two electrode 



August 20, 2003 



reconfigurable HOE stack for each pro D ected image, the size or errec 
exit pupils at the eyepiece optics may be increased. 



French Abstract 



rencn ADstract , ...ppj- <je 

nn decrit un dispositif de visiocasque (10) et un proceae ae 

reconf igurables (HOE^ jispog'itif Les HO^reconf igurables peuvent 
? Tenure 'piu ^pUr^s'l-Ples fonctiens opti ? ues ge „erale M „t 

d'electrodes (56). Cet hologramme est constitue d'un film 
phoiopoJymerique holographique associe a des ^ ista Zn a lTt e n revonse a 
L' hologramme presente des proprietes optiques qui ch angent en reponse a 
I- application d'un champ electrique. Des etages (28, 3 , 64 66, 68, 70) 
^=hTk= hof reconfiaurables sont mis en oeuvre pour dittuser 
fe S vemenfSes images monochromes projetees d'un generator d- image 
MB? soit vers l'oculaire droit soit vers l'oculaire gauche (24, 26) du 

; s , P our presenter une scene virtuelle en stereoscopie ou sous 
rorSe binoculaire P monochrome. En utilisant P lusie ^s ft^ges d' HOE 
reconfigurables pour chaque image pro D etee, on peut augmenter la taille 
de la pupille de sortie effective aux oculaires . 
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display system that utilizes a single image source. 



nr^rRT PTTON OF THE RELATED ART 

DESCRIPTION ™ dis pi ay systems generally rely on two image 

sourcefto vTsuaUy present right and left images of scenes to the right 

ft J user Typically, the first image source generates an 

imaae of Tscene ha to be viewed by the right eye of the user , 

whi?e ^second image source generates an image o^scene^hat^is to 

S.JlTSM.i^To dLpfay Scenes in a binocular mono forn . or in a 

stereoscopic form. In a binocular mono head - »°™ ted , ^"^^ of 
system? thJ images that are to be viewed by the right and left eyes of 
the user are the same perspective images of a scene. 

When these same perspective images are simultaneously... 

observes a binocular mono scene produced by ^V 1 *" and left ^f^re 
"in a stereoscopic head - mounted display system, the images that are 
to be viewed by the right and left eyes of the user are two slightly 
differenrperspective Images of the same scene. When these slightly 
different perspective images are simultaneously viewed, the us er 
observes a stereoscopic scene produced by the right and left 
perspective images . 
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In contrast to the headhunted fro- the hologrjphic diffuser. The^ 
virtual «ene can be vxewed^by th e the ding on the right 

.iS'SMi- ^ - pro^ted^tL virtual scene may be in a 
stereoscopic form or in a binocular mono form. 

in a modified configuration of the head-mounted... 

...image source, i.e., the image generator is required to ^P^JJ™^ 

t*e size of elective exit pupils at the eyepiece optics may. . . 
...projected images to minimize loss of both brightness and field of view 
perceived by the user . 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a schematic perspective view of a head... 
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English Abstract *i- er eoscopically projects a three 

A computer system (200-32^) stereoscopitdxiy y j ce 

^^^^^^^^^^^ 

is activated when the user's finger moves within the space to toucn tn 

projected icon. 



August 20, 2003 



""^""f concern, un s yS te„e i"^™^- S^^! g^'ftLS m ° de 
ster.oscopique 1 ■ image d'un «^»"?™ \ i Lteur UOOl'peut 

Tl • interieur de c.t espace tout .nob,, rv.nt a la fo ^ e ^« f * rlnatique 

SIW; WS a"n ^SMir^SiS J- .y-t-J-r. on 

1, de command des que l'objet physique went «uper le g** 

"U.*int.r« s .iun.^^ 

indique par une image d interlace 4 , nrecite. L'activation 

rtSi^rtSfTS^^^ Hn^cone dePl " e ~ 
doigt dans 1- espace jusqu'a toucher la projection de 1 icone. 

Main international Patent Class: G09G-005/00 
Fulltext Availability: 
Detailed Description 

Detailed Description 

has spent a life time touching physical 

This provides for an intuitive interface. 
Se iterloicoplc projector 200 can be any of several display 
^Ls^apable Sf displaying three ^nsional -ge^ om^ 

border 'Jo determine the corresponding position and orientation. In 

■ be used to determine the aforesaid positions and... 
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Publication Language: English 
Fulltext Availability: 

Detailed Description 

Claims 

Fulltext Word Count: 3098 



Eng An S LSac?fve desk system (10) comprising a horizontal desktop (14) 
-ULSdfsir^TU^ 
cSuSrl^ 

JoXto" a display projector coupled to the computer for promoting the 
pattern through the underside surface of the desktop surface to be 
visible on the top display surface. 



sStS: Te table de travail interactive, comprenant une surface de 
travS sensiblement horizontal, dont au moins une P artl * ^f^rface 
nartiellement la lumiere et comporte une surface infeneure; une surface 
sSoerieure de visualisation; un releveur de coordonnees qui peut etre 

! T t I^ionne par 1 • utilisateur, et emet une sortie mdiquant sa 
positionne et actionne p sur face de visualisation superieure; 

^or^ateT^ fecoi? If sor^inolcatrice de la position du releveur 
Te cordonnees et genere un motif derive au moins partiellement de cette 
sortre; eEun projecteur d'affichage couple a I'ordinateur pour propter 
le motif a t ravers la face inferieure de la surface de travail, de 
manTere qu'il soit visible sur la surface superieure de visualisation. 

International Patent Class: G09G-005/00 ... 
Fulltext Availability: 
Detailed Description 

"^'the pr^enf invention. In this embodiment the desktop image is created 
by projecting a preferably stereoscopic ..image onto a partially 
transmissive surface between the users eyes and the 

desktop with appropriate perspective and focus so that it appears to the 
user -to-be residing on the desktop itself In the preferred embodiment 
shown in° F fg ^the'transmissive surface is fixed to t, ; use, j head by 

=L:L?ly^ilao^rom ^Xni 2 ^ by 

available from Polhemous. 
It will be appreciated by... 
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Claims 

Fulltext Word Count: 6401 

sis kiSSSSSiS 

"seK'eyes so that th. in»ges are perceived stereoscopically and 

speakers positioned £ °™« Vo^L^Ier? ."ars U6, 18 lilt sound 

cnpaVprs oositioned rearward of the user s ears \j.±o, .7 w - 
can SfpeSSved to originate from a particular directs or object. 

rotative, en trois dimensions, a un bras articule (102) supportant 
'l'pffirhaae visuel (104). L'utilisateur introduit sa tete dans 
1'affichS VisSel 104 et s'empare des poignees de commande (14 4) avec 

visuel (104) abr^e deux affichages a cristaux liquides dirigeant des 
Images eparees vers les yeux de l'utilisateur afin de P'^r^une 
perception stereoscopique des images, ainsi que des ha ^"Parleurs places 
a l'avant des oreilles de l'utilisateur 115, 117 et d " f ^ ^leurs 
olaces a l'arriere des oreilles de l'utilisateur (116, 118 afin que le 
son puisse etre percu comme provenant d'une direction ou d'un ob D et 
particuliers . 

Main International Patent Class: G09G-005/00 
Fulltext Availability: 
Detailed Description 

Detai ob?ec e tni P ^r virtual world may be perceived to move relative to the 
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user . 



may 



instructs the processor to adopt -^^^J^S ^ ^ user may 

ScSSr 1 .'LEoT^^ « S j. £ PJ- ryof - ~ 

field of view. Through manual controls o sensors, *he user 
the control menu to perform various function,. For example,^ ^ ^ 
control menu the user may ^?^^«^ ft8ente A 0 the user m, 
S" omputeJ 0 such- t h^ a the r user sees a representation of his person m 
the virtual world . According to a second mode, the images are 
compuSfsuch that the user views the virtual world througha ^ 

virtual helmet as if .^"^^^U^nTiirtual WO 97/49081 
images are computed so that the user vie significant 
PPT/US97/10107 of mass of the apparatus is . . . yi. <j y „l 4 .^„ wii-h a 

supported by a boom and mechanically coupled... 

102 and the counterweighted pitch axis 108 allow... 

relatively large LCD displays in the visual display so that a large 
••portion of y tnr 9 user 's field of vision is occupied. The use of a 
counterweight boom located below the, . . 

of mass The configuration of the visual display can reduce the setup 
...of mass, ine C °»"9 U because the user may simply walk up to the 
time for each user , Because cne us »" * *\ , 
apparatus, bring his face into contact with the visual... 

...pair can significantly .improve the quality of the image and sound data 

transmitted to the user . 

The above-described exemplary embodiments are intended to be illustrative 
in all respects, rather than... 
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Claims 
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English Abstract virtual environment, robot or 

„ser is P°" t " n ^. on 2 |. SU '** C ; i rt„ a " e ^ ir o„»ent, robot or vehicle 

^S-TSMi to" inputs "~ » «»* <=°»«°1 < 2 '" U » B 

a second control region (26) . 

French Abstract Hi<mn«3itif de commande de mouvements (16) 

Cette invention concerne un dispositir ae ^uimiiduu ■ . . f d 

~ u„ .irtuel ^ »» ^ ; e„U. « * 

l'environnement virtuel. Un utilisateur se _ t J ^ 



le monde reel po\ 

L'environnement virtuei. un uu4i««« m " ri I";r 26) L'environnement 

pro^iennent de la premiere zone de comande (24) o» de la 

deuxieme (26) . 

Main International Patent Class: G09G-005/00 
Fulltext Availability: 
Detailed Description 

Detailed Description ^ aircraft _ objec t is to 

'"train Se nator and provide a near-real experience. The use of a 

Ifvi^ent- ^^L^S.^. Kim 

et al. . . 

Reality Designing Metaphor for Novice Users," describe a virtual 

cSSS -'floaty ,»ide a -1^- ' S»y 

SSK-Tn^T^rr SSt =Ter „."ne field of SS;^.? CTon. 
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"TV „ m SSS'l&MlinSS 1 (PCT/WO US9510798, 

duplication: "s 9429^83 19940824; OS 94328371 19941025; US 

"if! ShkkkkkhfV 

IE IT 11 MC £ P? SE BF BJ CF CG CI CM GA GN ML MR NE SN TD TG 
Main international Patent Class: G09G-005/00 
Publication Language: English 
Fulltext Availability: 

Detailed Description 

Claims 

Fulltext Word Count: 14538 

Kde-by-side .elation, the lower resolution i„,a ? e being provided by the 
2 a method of display includes forming a relatively lower 

at least a portion of the relatively higher resolution image. 

'"sStetf ^'visualisation comprenant un afficheur a resolution 

relativement haute destine a presenter des informations visuelles, et un 

"on"^ 

coniuque avec la lumiere incidente sur ce reflecteur, afin de tormer 
° a visualiser, et 1 ■ image a haute resolution etant creee sans 

l 'intervention de l'optique de focalisation. Un precede d affichage 
consist firmer un image reelle a resolution relativement faible, a 
rerlecnL I'^mage en direction de 1'oeil de 1 • observateur a former une 
imaae a resolution relativement haute, et a diriger cette dermere en 
SSctLn de 1'oeil de 1 ■ observateur, de sorte qu'au moms une par » de 
rlmage a resolution relativement faible entoure au moms une partie de 
1' image a resolution relativement haute. 

Main International Patent Class: G09G-005/00 
Fulltext Availability: 
Claims 

Claim an eyeglass frame, and/or other support structure in the form of 
"housina 61 The HMD system I ' includes two optical viewing systems 4 

Ale tie one described above with respect... to provide appropriate 

Positioning of the output ports 42 relative to e eyes user . 

Alternatively, the system I • ■ can be positioned m a relatively fixed 

location, and the eyes... 
...of the eye. This can be accomplished in the manner illustrated in Fig. 

?he HMD system I can be compact and still provide comfortable viewing 
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distance of about twenty inches... 

.as was mentioned above, for example, with respect to the focusing optics 
20. In the HMD system 1 of Fig. 6 the viewer is previa 
virtual image 23a of . . • 

.equivalent techniques) for Providing light to the ^"o^Mch'can 

illustrates another embodiment of HMD system 100, fe ure 

oe used with the several embodiments desc " be tstem 10 of'the HMD 

light is not necessarily lost to the optical «yate» 10 of the HMD 

system 100. Rather, such light may be used to increase tne or g 

the light . . . 

v, 1 1 Tt will be appreciated that the HMD 

the brightness of . . . 
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Detailed Description 

Claims 

Fulltext Word Count: 8645 
English Abstract 

The invention provides a method for a subscriber television system (Fig. 
1A) client device (130) to provide a three-dimensional user interface 
comprising a virtual reality media space. The three-dimensional user 
interface allows the user to navigate a three-dimensional environment, 
participate in activities, and interact with other users. The 
three-dimensional user interface enables the user to associate personal 
characteristics with an avatar which represents the user in the system, 
such personal characteristics comprising a symbol, a picture, and video. 

French Abstract 

L* invention porte sur un dispositif client (130) pour systeme de 
television d'abonne servant d' interface utilisateur tridimensionnelle 
presentant un espace de media a realite virtuelle. L f interface 
utilisateur permet a 1 1 utilisateur de naviguer dans un environnement 
tridimensionnel, de participer a des activites et d'interagir avec 
d'autres utilisateurs . L 1 interface utilisateur tridimensionnelle permet a 
1 ' utilisateur d'associer ses caracteristiques personnelles , qui 
comprennent un symbole, une image et une video, a un avatar le 
representant dans le systeme. 

Legal Status (Type, Date, Text) 

Publication 20030109 Al With international search report. 

Examination 20030515 Request for preliminary examination prior to end of 

19th month from priority date 

Main International Patent Class: G09G-005/00 
Fulltext Availability: 
Detailed Description 

Detailed Description 

... MS-UI I 00 implementations. 
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FIG . 2 is an. illustration of a snapshot of the VR -MS-UI 1 00 (FIG. 1 A) 
in accordance with one embodiment of the current invention. In the 
embodiment of the current invention depicted in FIG. 2, the VR -MS-UI 
100A consists of a vehicle navigating . ..1C) to maneuver around in the 
3D Landscape 290 by pressing the up button to move forward and similar 
buttons for other directions. The Client Command Device 2 160C (FIG. 
1C) could also be other instruments, such as a joystick or a... the 
functions desired the user could, in part, make use of the controls on 
the VR Dashboard 210. A World Map 220 is a feature that indicates the 
user's position within the entire current 3D virtual world in 
relation to other attractions and elements inside the world. This feature 
allows the user... the user chooses to navigate the 3D Landscape 290 in 
the virtual vehicle, then the user can control the speed of movement 

of the virtual vehicle with the Throttle 240 on the VR Dashboard 21 0. 

Moving the Throttle 240 forward In a non-limiting example, the user 
would be enabled to move the Throttle 240 through the use of a Client 

Command Device ...down arrow to throttle down. Selection of the 
Throttle 240 or other elements of the VR Dashboard 2 10 could occur 
through any of a variety of methods. For example ... IC) . Another example, 
among others, includes highlighting, or bringing in focus, various 
elements of the VR Dashboard 2 1 0 in a looping manner among all 
selectable items such that a... 
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Fulltext Word Count: 7663 
English Abstract 

A virtual simulation system generates an "image of a virtual control on 
a display that may be a heads-up-display in a vehicle. The system uses 
three-dimensional range finding data to determine when a user is 
sufficiently close to the virtual control to "manipulate" the virtual 
control. The user "manipulation" is sensed non-haptically by the system, 
which causes the displayed control image to move in response to user 

manipulation . System output is coupled, linearly or otherwise, to an 
actual device having a parameter that is adjusted substantially in 
real-time by user-manipulation of the virtual image. System generated 
displays can be dynamic and change appearance when a user's hand is in 
close proximity displays can disappear until needed, or can include menus 
and icons to be selected by the user who points towards or touches the 
virtual images. System generate images can include representation of the 
user for use in a training or gaming system. 

French Abstract 

L f invention porte sur un systeme virtuel de simulation elaborant 1 1 image 
d f une commande virtuelle sur un ecran pouvant etre le dispositif tete 
haute d'un vehicule. Ledit systeme utilise des donnees de telemetrie en 
3D pour determiner quand 1 1 utilisateur est suffisamment rapproche de la 
commande virtuelle pour pouvoir la "manipuler". Les "manipulation" de 
1 ' utilisateur sont detectees non haptiquement par le systeme, ce qui 
provoque le deplacement des images de commande en reponse a ladite 
manipulation. Les signaux de sortie du systeme sont couples, lineairement - 
ou autrement, a un dispositif reel presentant un parametre s'ajustant 
sensiblement en temps reel lors de la manipulation de 1 1 image virtuelle 
par 1 ' utilisateur . Les images produites par le systeme peuvent etre 
dynamiques et changer d'apparence lorsque l ! une des mains de 
1 ' utilisateur se trouve a proximite immediate. Les images peuvent 
disparaitre jusqu'a ce qu'un les utilise, ou peuvent comprendre des menus 
et des icones selectionnables par 1 1 utilisateur en pointant sur une image 
virtuelle ou en la touchant. Les images produites par le systeme peuvent 
comprendre une representation de 1 1 utilisateur utilisable dans des 
systeme de jeu ou d 1 entrainement . 

Legal Status (Type, Date, Text) 

Publication 20020815 Al With international search report. 

Publication 20020815 Al Before the expiration of the time limit for 

amending the claims and to be republished in the 
event of the receipt of amendments. 

Examination 20030213 Request for preliminary examination prior to end of 

19th month from priority date 

Main International Patent Class: G09G-005/00 
Fulltext Availability: 
Claims 

English Abstract 

A virtual simulation system generates an image of a virtual control on 
a display that may be a... 

...control. The user "manipulation" is sensed non-haptically by the system, 
which causes the displayed control image to move in response to user 

manipulation . System output is coupled, linearly or otherwise, to an 
actual device having a parameter that... 

Claim 

said image on said 
display; 

(c) determining non-haptically from data sensed at step (b) , user 
intended movement of said image of said control ; and 

(d) outputting a signal coupleable to said actual device to control 
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parameter as. a function of sensed user intended movement of said 
image of said control . 

1 5 

2 The method of claim 1 , wherein at step (a), said display is... 



19/5, K/3 (Item 3 from file: 349) 

DIALOG (R) File 34 9: PCT FULLTEXT 

(c) 2003 WIPO/Univentio. All rts. reserv. 

00779767 +*Image available** 
VIBROTACTILE HAPTIC FEEDBACK DEVICES 
DISPOSITIFS VIBROTAC TILES A RE TOUR TACTILE 

Patent Applicant /Assignee : Q , ni n <3 US 

IMMERSION CORPORATION, 801 Fox Lane, San Jose, CA 95131, US, Ub 
(Residence), US (Nationality) 

Inventor(s): qa^ii nq 

SHAHOIAN Erik, 738 Maud Avenue, San Leandro, CA 94577 US^ 
MARTIN Kenneth, 4240. Suzanne Drive Palo Alto, CA 94 306, US, 
SCHENA Bruce, 414 Pope Street, Menlo Park CA 94025, OS, 
MOORE David,- 318 West Oakwood Boulevard, Redwood City, CA 940bl, Ub, 

"llCKMX^uf f (agent), Hickman Coleman t Hughes, LLP, P.O. Box 52037, 
Palo Alto, CA 94303, US, 

Patent and Priority Information (Country, Number, Date . 

o WO 200113354 Al 20010222 (WO 0113354) 

looUcation- WO 2000US22761 20000818 (PCT/WO US0022761) 

^5 Stes^ArAG^^'AT^uTz'riB BG BR BY BZ CA CH CN CU CZ DE 

SD SE SG SI SK SL TJ TM TR TT TZ UA UG UZ VN YU ZA ZW 

(EP) AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE 
OA) BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG 

(AP) GH GM KE LS MW MZ SD SL SZ TZ UG ZW 

(EA) AM AZ BY KG KZ MD RU TJ TM 
Main International Patent Class: G09G-005/00 
International Patent Class: G09G-005/08 
Publication Language: English 
Filing Language: English 
Fulltext Availability: 

Detailed Description 

Claims 

Fulltext Word Count: 13378 

the housing, and a mass. In some embodiments, the mass ca n be °J c ^ lated 
by the actuator and a coupling between the actuato f h ^ d J^\™ a ^ r ° ed 
between the mass and the housing has a compliance that can be var led 

pSpniricitv allows vibrotactile sensations having different magnitudes 
"for a g'enVrive signal to be output to the user gasping the housing. 

French Abstract 
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L* invention concerne un procede et un dispositif servant a commander 
l'intensite et la frequence de sensations vibrotactiles de dispositif s a 
retour tactile. Un dispositif a retour tactile, tel qu'un organe de 
commande de tablette de jeu, une souris, une telecommande, etc., comprend 
un corps (14) tenu par 1 ' utilisateur , un actionneur (30) couple au corps 
et une masse. Dans certaines formes de realisation, 1' actionneur peut 
faire osciller la masse, et un couplage entre 1' actionneur et la masse ou 
entre la masse et le corps presente une compliance pouvant etre modifiee. 
Une modification de la compliance permet de produire sur 1 1 utilisateur 
tenant le corps des sensations vibrotactiles de differentes intensites 
pour un signal de commande donne . Dans d'autres formes de realisation, 
1' actionneur est un actionneur rotatif et la masse est une masse 
excentree pouvant etre tournee par 1' actionneur autour d f un axe de 
rotation. La masse excentree presente une excentricite pouvant etre 
modifiee par rapport a 1 ' axe de rotation pendant que la masse tourne. Une 
modification de 1 1 excentricite permet de produire sur 1 1 utilisateur 
tenant le corps des sensations vibrotactiles de differentes intensites 
pour un signal de commande donne. 

Legal Status (Type, Date, Text) 
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Publication 20010222 Al Before the expiration of the time limit for 
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Detailed Description 

14. For example, the host application program can be a video or 
computer game, medical simulation , scientific analysis program, 
operating system, graphical user interface, or other application program 
that utilizes force... 

...For example, when a user-controlled vehicle collides with a fence in a 
game or simulation , a vibration can be output to the user to enhance 
the interactive experience of the collision. Similarly, when a user - 
controlled cursor moves onto .another object such as an icon or text 
heading, vibrations can be used to... 
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Eng A iS forS St feeSbac k interface (200) is coupled to a host computer (202) 

generate the force. 

ZlToiTnl a P ll tefr^""" u„ co»ra„t ««„t envoye , 1. bobine afm de 

produire la force. 
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Fulltext Availability: 
Detailed Description 

^'"controlSnrthe displayed environment. The computer updates the game 
user utilizing the display screen and, typically, audio... 



obiect of the interface device, such as a joystick, mouse, steering 
"wSr s^lus, etc., and .oves the objert^P^ided^degrees of 

Mi SpuSTSi force^aSins Strolled by the 
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English Abstract „ lloled to a host computer (18), 

A force feedback interface device (14), coupled to a no £ 
includes a manipulandum (12), such as a mouse, ^at .s moveabl e xn 

Soffih. -use (12) in the local frame ( ^^tj^ho. ^(18,^ ^ 



control 
French Abstract 



S^ijS U2. , «! ^okKn^rs »f SSTdE S cadre 

ss^^Iret t Se d L^^'"t^s l dr^^r?;:^?^^5^e » tre »n 
- ££ frrs^^L^Keier d er IO "ti^r^"aeLS„"rri:y„rer^t?r 

d'ameliorer la .commande du curseur. 
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video game, a 

14 



user can be controlling a computer player or vehicle in the virtual 
environment by manipulating the mouse 12. The computer syste ter 
no* it ion of the mouse with sensors as the user moves it. i> v 

surface such as an edge of a window, a... 
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^Torcf f eeoJack mouse interface ^vice as shown in the Figure connected 
to a host computer and providing realistic force feedback to a user^ The 
mouse interface device (11) includes a mouse object (12) and a l^kage 
TaO) coupled to the mouse that includes a plurality of member s (46 50) 
rotataWy coupled to each other in a planar closed-loop linkage, two of 
til Ambers coupled to ground and rotatable about the same axis (52) Two 
ari-nai-ora (64) preferably electromagnetic voice coils, provide torces 
tS two degrees of freedom of the planar workspace of the mouse object. 
Each of the actuators includes a moveable coil portion ("a) integrated 
wttn one of S members of the linkage and a magnet portion (90a) coupled 
to the around surface through which the coil portion moves. At least one 
sensor (62) is coupled to the ground surface that detects movement of the 

e and provides a sensor signal including information from which a 
posSon of S mouse object in the planar workspace can be determined. 

Fre un h dX P os"if d' interface de souris a retour de force est connecte a un 
orSina?eur hote et fournit a 1 ■ utilisateur un retour de force realiste. 

p^n^ 

E lo- Krm-un 'y^ art Icuie^n £3?^^ - de s 

eectiomagnefiques mobiles, produisent des forces corres pendant aux eux 
deqres de liberte de l'espace de travail plan de la souris. Ch a un des 
dlspositifs d'actionnement comprend une portion bobine mobile (64a), 
an des elements du systeme articule et une portion aimant 

de LSion- com'renLt une information a partir de laquelle la position 
de la souris peuf etre determinee dans 1'espace de travail plan. 
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Detail ! d w ^ras P ro?ary degrees of freedom. For many applications, mouse 12 
... as well as rotary ^grees preferred embodiment, the user 

need only be moved n "'J™^ ,3^, mucn like a traditional 

manipulates mouse 1^ j." . p ° ^ . „^ Ka HiQniavpd e a , imaqes of 
j m(ar iirflV simulation, can be aispiayea, e.y., xmav ^ 

mouse, and the . . .medical sirauidtiui , obiect 12 moving 

tissue and a representation of a manipulated user object m y 

through the tissue, etc. 

There are two primary " control paradigms" of operation for mouse system 
IS! position control and rate control. Position control is... 
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^A^eS'apparatus (50, comprises a visual display (104) "ans.ittin, 

heaS into the visual display (104) and grasps the control grips U^) 

relaKSe to the user and the virtual world and objects. The visual 
display (M houses two LCD displays which direct separate images to the 
use^s'eyes so that the images are perceived stereoscopically and 
speakers positioned f o™»^ of the «sex^3 ea^s Cll , 1 and the 

S'S-peSiiirS orlgin^e'.rora^rrtic^rr Erection or o bj ect. 
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rotative en trois dimensions, a un bras articule <1°?) support ant 
"affichage visuel (104). L'utilisateur introduxt sa tate^an- ^ 
1-affichage visuel (104) ^ . - ' ^j^poxgnees £ I virtuel 

kfiSat^ 

i /m/M a hritp dpux affichaqes a cristaux iiquiutss uliA y^ 
visuel (104) abnte aeux aiu ' y ... . fin de permettre une 

»o'"uiLe :S p.ro» co»m. prevent <Tuna direction 0» d'un objet 

particuliers . 

Main international Patent Class: G09G-005/00 
Fulltext Availability: 
Detailed Description 

D6tai the Ter'fllZ with the Sdi.ensional world Additional signals can be 
••transmitted from the handle 14 4 of the apparatus 50 ^f ^tU 
Seward, -^^ a ^r^%^^oZ translation _t 

controls can. . . 

...of the visual display 104, and may be limited only by the definition of 
the virtual world . 

As will be readily appreciated by those skilled in the art, the present 

invention provides. . . 
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^meVnTand'appar.tus for providing ,.», to J «J« operating 

a human/computer i"t« £ »« (1, > ^ .soec? a computer implemented 

C °SrSu;,«s1heTn fraction"^ SI.S * «t S displayed to a user 
method simulates tne int objects by manipulating a physical 

S°ect""« S ot n° nterfardevtce A» . The physical object provides 
22Ir feedback to the user which imparts a physical sensation 
corresponding to the interaction o£ the computer simulated objects. 



^Tettf invention se rapporte a u„ P™«^,* -tKs.t.u/actionnant u„ 

^^™^£*^<°£^^ « f -"Td: s ce^r c une 

donne ine sensaJionVysique correspondant a 1 ■ interacts des ob D ets 

simules par ordinateur. 

Main International Patent Class: G09G-005/00 
Fulltext Availability: 

Detailed Description 

Claims 

^'^SS^n-th a table The inter, ace device e,^ 

player to interact with a sporting simulation . 

An embodiment of the present invention for a sporting simulation includes 
a paddle simulated object... of the present 

Kgures^a-c illustrate the use of an interface device to move a user 

^rs-o T f he fre= r^ssa s ~ 

■ simulation or application program implemented by the host computer. User 
object 34 can be a joystick... 

A user object 34 and interface device 14 for simulating a racquet in a 
•' sporting simulation is described in greater detail with respect to 
Figure 19. 

, kpr ob , ect 34 is preferably... using a host command, as explained below. 
in the described example, paddle object 220 is controlled by a player 
S alosSon control paradigm such that the movement o pad d e 

JL+*rZ ??n is directly mapped to movement of user oDDect j«. ^ n 

object can be given a weight as well. 

in still another... directly onto the planar workspace of the^nterface 
device 506 and 506' such that the user is moving the user object 

JS^lfo! « StS acf Silce^reSIy iSS a projected image of a 
wall, and the . . . 
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...handle could not be moved through. This allows a greater sense of 
immersion into the simulation . 

FIGURE 17 illustrates a "reverse projection" embodiment 540 of the game 
apparatus of the present ... embodiment allows a player object to move 
within a simulation "space" based on a rate control paradigm 
controlled by the user f s movement on stage 568. For example, in a 
simulated tennis game, the player could lean left... 

Claim 

is trapped as if said paddle is a flexible sling, and wherein when 
said trap command is removed while said ball object is moved by said 
user in an approximately circular path, said ball object is disengaged 
and moved tangential to said circular path away from said paddle 
object. 

53 A method as recited. . . 
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English Abstract 

A motion control device (16) for a virtual environment , robot or 
vehicle. The controller allows the user enough movement in the real 
world to create a sense of reality and presence in the virtual 
environment . A user is positioned on a surface and is able to move 
within multiple control regions (24, 26). The virtual environment , 
robot or vehicle responds differently to inputs from a first control 
region (24) than from a second control region (26). 

French Abstract 

Cette invention concerne un dispositif de commande de mouvements (16) 

pour un environnement virtuel, un robot ou un vehicule. Le dispositif de 

commande donne a 1 1 utilisateur une liberte de mouvement suffisante dans 
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ie „onde reel pour creer - j»pressio„ de realise et de P^efpeut 
l'environnement virtuel. Un utlll * at *^ *® ",34 26) L ' env i r onnement 
56 f Pl rtl tSll P "l*l™l r p^Xt^ere^ent - entrees selon 
Provrennen? de 1, pre„ier/zo„e de co»»ande (24, o« de la 

deuxieme (26) . 

Main International Patent Class: G09G-005/00 
Fulltext Availability: 

Detailed Description 

Claims 

Enf :H;r E^sss,' 1 :^ - en-r—^ne - 

^idlo ' .-at " nse ^reality - P~ - - aSf^ove 

environment . A user is posuiuneu virtual environment , 

region (24) than from... 
De "^ Srtuafenvironrcent is depicted ov one or ^Jl^ ^f 

sssot *ssr& 

environment include a keyboard, a pointing device and... 

desired. 

Sophisticated military flight simulators use an aircraft vehicle... 
...Held Miniatures." A hand-held miniature graphical "P^^J^f 8 

virtu :Lr t ir zr n v s usr r d t L^^ic<^«.«t t .San^rSi«.if 

-~^ lh - &r — s -STc^ n ?^?hf minl^rf 
arlpSS change d provide tL effect of the user shrinking into the 
Sniature or ?he miniature expanding to the enlarged virtual 

Sit The motion then occurs. Then the graphics change^o 
provide the effect of the user growing or the virtual environment 
shrinking. 

Twata et al describes virtual perambulator prototypes in (i) "Virtual 

or other graphical .„viror»ent An op«.tor £ov -P^ n « t »°^ 1 
the'Input'devSe JS^-SSf^i-inroS^ inputs to control what 



IS . 



for remote control of a robot... 

.. example, the input device surface 20 is de •'(■^."SS^SS"" 
% • S S f l! ?jf ^cessor^cei^rinput'Snars fro V he sensors 

with a Sst transformation function. Movement of the operator 
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^^^-^^^^^ " operator 
P^Son'wi^n the first control zone is directly transposed to 
movement within the virtual environment or of a robot or veni 
For example a I to I ratio of movement... 

size shlpe and location of each control... s center of gravity. 

This is taken as the operator position. The location of the °P«ator 
position with respect ?o the control zones dete ™- eS w S in °^ a ^ er 
*. i-^ hp nrocessed Operator movements within tne innex 

^r e of zonr 2 4°ar e e £Sce.sed according to one transformation function 
(e g , direct positional translations].). For... 

amount equal to a gain factor times the amount .oved in the inner 

«««*-r-ni 7 nne Operator movements within tne outer cohwu^ 

~f processed" acSrSng to another transformation function. For example, 

a movement in a . . . 

at a velocity equal 10... 

the operator pushes the ring 134 away from 

"^An input apparatus (16) which an operator interacts with to .control 
'movement within a virtual environment , the appara us co.px^ 
a surface (20) upon which an operator is positioned, the surface denning 

...a processor (18) receiving the output signal for processing sensor 
output to , . 

determine a current operator position; and 

wherein movement of the operator within a first control zone U4) 

of the multijli concentric control zones causes a first response in the 

virtual environment based upon a first transformation function, and 

wSetn movement of the operator within a second *° ntr °l *° nf ? 

of the multiple concentric control zones causes a second response in the 

vLtual environment based upon a second transformation function which 

differs from the 

first transformation function. 

2 The... the surface and compares such movement pattern with prestored 
movement pattern signatures to identify the movement pattern. 

10 An input apparatus (16) which an operator interacts with to control 

BnoM „ t within a virtual environment , the apparatus comprising: 
a Z^Tce (^ upon wSch an operator is positioned, the surface defining 

plurality of sensors, the processor determining a current operator 
position relative to the plurality 

of control zones from the sensor output signals, and 
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from the first transformation function. 
IL The. . . 

.from a time sequence of the output signals. 

12 An input apparatus (16) which an operator interacts with to control 

movement Within a virtual environment , the apparatus W^J; 
a surface (20) upon which an operator is positioned, the surface dermi g 

wherein the first control zone is concentrically inward relative to tne 
second control zone ... steepness corresponds to velocity within the 

virtual environment. 

16 An input apparatus (16) which an operator interacts with to control 

movement within a virtual environment , the a PP a "tus compri sing: 
a surface (20) upon which an operator is positioned, the surface defining 

the operator position within the first control zone translates directly 
'to positional change within the virtual environment , wherein movement 
of tte operator into contact with the ring deflects a portion of the... 

..wherein deflection of the ring into the second control zone translates 
velocity within the virtual environment . 

17 An input apparatus (16) which an operator interacts with to control 

mnTOmsnt within a virtual environment , the apparatus comprising: 
a plJtrorm having a surface (20) upon which an operator is positioned... 

..control zone of the multiple concentric control zones translates 
directly to positional change within the virtual environment , wherein 

n prater position within a second control zone of the multiple 
concentric control zones translates to change in velocity within the 
virtual environment , and wherein the first control zone is 
concentrically inward relative to the second control zone... 

the surface and compares such movement pattern with prestored movement 
pattern signatures to identify the movement pattern. 

^™ininfnr; t t a e nd°Lco„d... Z cL directly translating the change 
in operator position to a change in position within the 

fSTS.i.SSrSS.'.J'tu, operator position within the second 
control zone, translating... 

...which an elevated ring (134) is positioned 
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of the ring into the second control... 

..the ring into the second control zone translates to a change in velocity 
within the virtual environment . 

27 A method for an operator to control motion within a virtual 
environment, wherein the... 
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^hlS/Sputer interface device ,14, which has a physical object ,34 , , 
such as a joystick, controlling a gr aphi cal ££f " £ ? om th e 

SE^uter^the inter^oevic "to app ff ce s ensat onto * . 

ohvsicallv inrorms the user of the graphical objects encountered by the 
cuSor with" the GDI. A microprocessor (26) local to the interface 
apparatus can be used to control forces on the physical object. 

""u'invtntiofporte sur un precede, ainsi que sur I'appareil 

corresponding destines a procurer un retour de force a un utrlisateur 
Srsant fonctionner un dispositif a interface ho^e-machrne, ■ 

olTets graphiques et les fonctions du systeme Sexploitation de 
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1' interface graphique homme -ma chine, est determinee par un emplacement du 
curseur dans 1'IUG par rapport a des cibles associees aux objets 
graphiques. Au nombre de ces derniers figurent des icones, des menus se 
deroulant et des elements de menu, des barres de defilement ( "glisseurs" ) 
ainsi que des boutons. La sensation de force, qui aide 1 1 utilisateur a 
choisir une fonction du systeme d ' exploitation, peut egalement const ituer 
une information concernant les objets graphiques rencontres par le 
curseur a I'interieur de 1'IUG. II est possible d ! utiliser un 
microprocesseur relevant du dispositif a interface mais separe de 
l'ordinateur hote pour agir sur des forces s'exercant sur I'objet 
physique . 

Main International Patent Class: G09G-005/00 
Fulltext Availability: 
Detailed Description 

Detailed Description 

paddle 360 can be controlled by host computer system 12, and paddle 
362 can be controlled by the user by physically manipulating the 
user object. Ball 352 can be moved on display screen 20 according to 
simulated physical parameters, such as velocity, acceleration, gravity, 
compliance . . . 
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SYSTEME ENTREE/SORTIE MECANIQUE D 1 UN FAIBLE COUT POUR DES SYSTEMES 
INFORMATIQUES 
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Priority Application: US 95560091 19951117 
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SE 

Main International Patent Class: G09G-005/00 

Publication Language: English 
Fulltext Availability: 

Detailed Description 

Claims 

Fulltext Word Count: 31888 
English Abstract 

A method and apparatus for interfacing the motion of a user-manipulable 
object (44) with an electrical or computer system (16). A gimbal 
mechanism (38) is coupled to the user object (44), such as a joystick or 
a medical tool, with at least two degrees of freedom. The gimbal 
mechanism (3B) preferably includes multiple members, at least two of 
which are formed as a unitary member which provides flex between the 
selected members. An actuator (126) applies a force along a degree of 
freedom to the user object (44) in response to electrical signals 
produced by the computer sytem (16) . A sensor (42) detects the position 
of the user object (44) and outputs sensor signals to the computer system 
(16). Another embodiement includes a local processor (410) separate from 
a host computer (16) for communicating with the host computer (16) and 
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controlling the forces output by the actuators according to a processor 
subroutine selected in accordance with a host command, sensor signals, 
and timing information. Another embodiment of the interface apparatus 
(14) uses voice coil actuators (126) that produce forces in either linear 
or rotary degrees of freedom using currents applied in a magnetic field. 
French Abstract 

Cette invention concerne un procede et un appareil permettant d' assurer 
1* interface entre un objet (44) manipule par un utilisateur et un systeme 
electrique ou informatique (16) . Un mecanisme a cardan (38) est couple a 
un objet utilisateur (44), tel qu'un manche a balai ou un instrument 
medical, tout en conservant au moins deux degres de liberte. Le mecanisme 
a cardan (38) comprend de preference plusieurs elements dont deux au 
moins consistent en un element unique assurant la flexion entre les 
elements choisis. Un dispositif d 1 actionnement (126) exerce une force sur 
l'objet utilisateur (44) selon un degre de liberte et en reponse a des 
signaux electriques generes par le systeme informatique (16) . Un capteur 

(42) detecte la position de l'objet utilisateur (44) et envoie des 
signaux de capteur au systeme informatique (16) . Dans un autre mode de 
realisation, on utilise un processeur local (410) distinct de 
l'ordinateur hote (16) qui va entrer en communication avec ce dernier 

(16) et regler les forces exercees par les dispositif s d' actionnement 
d'apres une sous-routine du processeur, laquelle est definie en fonction 
d'une commande de l'hote, des signaux des capteurs et des donnees de 
synchronisation. Dans un troisieme mode de realisation de cette 
invention, un appareil d 1 interface (14) comprend des dispositif s 
d' actionnement (126) a bobines mobiles, lesquels vont exercer des forces 
selon le degre de liberte lineaire ou selon le degre de liberte en 
rotation a I'aide de courants appliques dans un champ magnetique. 

Main International Patent Class: G09G-005/00 
Fulltext Availability: 
Detailed Description 

Detailed Description 

degrees of freedom. 

Joystick 112 can be used in virtual reality simulations in which the 
user can move the joystick to move a vehicle, point to objects, 
control a mechanism, etc. For example, a user can view a virtual 
environment generated on a computer screen or in 3D goggles in which 
joystick 1 12 controls... 
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Claims 

Fulltext Word Count: 32511 
English Abstract 

A method and apparatus for providing force feedback to a user operating 
a human/computer interface device (14) and interacting with a 
computer-generated simulation (20) . In one aspect, a computer implemented 
method simulates the interaction of simulated objects displayed to a user 
who controls one of*, the simulated objects by manipulating a physical 
object (34) of an interface device (14). The physical object provides 
force feedback to the user which imparts a physical sensation 
corresponding to the interaction of the computer simulated objects. 

French Abstract 

Cette invention se rapporte a un procede et a un appareil servant a 
produire une force de retour a destination d ! un utilisateur actionnant un 
dispositif d' interface homme/ordinateur (14) et interagissant avec une 
simulation generee par ordinateur (20) . Dans un aspect de cette 
invention, uh procede mis en oeuvre par ordinateur simule 1 ' interaction 
d'objets simules presentes a un utilisateur qui commande 1 1 un des objets 
simules en manipulant un objet physique (34) d f un dispositif d' interface 
(14). L'objet physique fournit a 1 ' utilisateur une force de retour qui 
donne une sensation physique correspondant a 1 1 interaction des objets 
simules par ordinateur. 

Fulltext Availability: 
Claims 

Claim 

is trapped as if said paddle is a flexible sling, and wherein when 
said trap command is removed while said ball object is moved by 
• said user in an approximately circular path, said ball object is 
disengaged and moved tangential to said circular path away from said 
paddle object. 

53 A method as recited. . . 



August 20, 2003 



30/5, K/2 (Item 2 from file: 349) 

DIALOG (R) File 349: PCT FULLTEXT 
(c) 2003 WIPO/Univentio. All rts. reserv. 



00401876 **Tmage available** 
VIRTUAL MOTION CONTROLLER 

DISPOSITIF DE COMMANDE DE MOUVEMENT S VIRTUELS 

Patent Applicant/Assignee: 

UNIVERSITY OF WASHINGTON, 
Inventor ( s) : 

WELLS Maxwell J, 

MANDEVILLE Jon, 

FURNESS Thomas A, 

PULKKA Aaron K, 

LAMAR Michael, 

ATEN Jason, 

Patent and Priority Information (Country, Number ,* Date ) : 
Patent: WO 9742620 Al 19971113 

Application: WO 97US7419 19970502 (PCT/WO US9707419) 

Priority Application: US 9616954 19960506 

Designated States: AL AM AT AU AZ BA BB BG BR BY CA CH CN CU CZ DE DK EE ES 
FI GB GE HU IL IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MD MG MK MN MW 
MX NO NZ PL PT RO RU SD SE SG SI SK TJ TM TR TT UA UG UZ VN GH KE LS MW 
SD SZ UG AM AZ BY KG KZ MD RU TJ TM AT BE CH DE DK ES FI FR GB GR IE IT 
LU MC NL PT SE BF BJ CF CG CI CM GA GN ML MR NE SN TD TG 

Main International Patent Class: G09G-005/00 

Publication Language: English 

Fulltext Availability: 
Detailed Description 
Claims 

Fulltext Word Count: 9356 



English Abstract 

A motion control device (16) for a virtual environment, robot or 
vehicle. The controller allows the user enough movement in the real, world 
to create a sense of reality and presence in the virtual environment. A 
user is positioned on a surface and is able to move within multiple 
control regions (24, 26) . The virtual environment, robot or vehicle 
responds differently to inputs from a first control region (24) than from 
a second control region (26) . 

French Abstract 

Cette invention concerne un dispositif de commande de mouvements (16) 
pour un environnement virtuel, un robot ou un vehicule. Le dispositif de 
commande donne a 1 1 utilisateur une liberte de mouvement suffisante dans 
le monde reel pour creer une impression de realisme et de presence dans 
1 ' environnement virtuel. Un utilisateur se trouve sur une surface et peut 
se deplacer dans plusieurs zones de commande (24, 26). L 1 environnement 
virtuel, le robot ou le vehicule repondent differemment aux entrees selon 
qu'elles proviennent de la premiere zone de commande (24) ou de la 
deuxieme (26) . 

Fulltext Availability: 
Detailed Description 

Detailed Description 

... virtual environment including, simulation environments, game 

environment, virtual reality environments or other graphical environment. 
An operator provides inputs to control movements .and enter 
commands in the virtual environment through the input device 16. The 
processors 18 process the inputs to control what is... 

...display device 14 provides visual feedback to the operator. Although 
other mechanisms for feeding back virtual environment information can 
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be implemented, this invention addresses the input device 16 the operator 
controls to provide input to the host processing system 12. Although a 
virtual environment host is described, the input device also is used 
for remote control of a robot or vehicle via direct view or via a 
displayed view. 
The. . . 

...example, the input device surface 20 is designed to resemble a surfboard 
for a surfing virtual environment . Fig. 2 shows a circular surface 
20. ...24, 26. The processor 18 receives input signals from the sensors 
22 to determine an operator position on the surface 20. Movement of 
the operator position within a first control zone 24 is processed 
with a first transformation function. Movement of the operator 
position within a second control zone 26 is processed with a second 
transformation function. The specific transformation function may vary 
depending on the virtual environment or remote control 
implementation. In a specific embodiment, movement of the operator 
position within the first control zone is directly transposed to 
movement within the virtual environment or of a robot or vehicle. 
For example a I to I ratio of movement... 

. . .movement in the first control zone is implemented as a 5 foot movement 
in the virtual environment or of the robot or vehicle. The relative 
size, shape and location of each control... 



